A null space property approach to compressed 

sensing with frames 



Xuemei Chen, Rongrong Wang 
Department of Mathematics 
University of Maryland, College Park 
Email: {xuemeic, rongwang}@math. umd.edu 



o 



CZ3 

o 



> 

o 

(N 
O 
m 



Abstract — An interesting topic in compressive sensing concerns 
problems of sensing and recovering signals with sparse repre- 
sentations in a dictionary. In this note, we study conditions of 
sensing matrices A for the I 1 -synthesis method to accurately 
recover sparse, or nearly sparse signals in a given dictionary 
D. In particular, we propose a dictionary based null space 
property (D-NSP) which, to the best of our knowledge, is the 
first sufficient and necessary condition for the success of the 
, I 1 recovery. This new property is then utilized to detect some 
of those dictionaries whose sparse families cannot be compressed 
universally. Moreover, when the dictionary is full spark, we show 
that AD being NSP, which is well-known to be only sufficient 
for stable recovery via I 1 -synthesis method, is indeed necessary 
as well. 

I. Introduction 

Compressed sensing concerns the problem of recovering a 
sparse signal xq £ C d from its undersampled linear measure- 
ments y = Axo <G C"\ where the number of measurements m 
is usually much less than the ambient dimension d. A vector 
■ is said to be k-sparse if it has at most k nonzero entries. 
The following linear optimization algorithm, also known as the 
Basis Pursuit, can reconstruct xo efficiently from a perturbed 
observation y = Axq + w where 1 1 tt? 1 1 2 < e ©El: 



x = arg mm 

xeR d 



1, subject to \\y - Ax\\ 2 < e. (1) 



A primary task of compressed sensing is to choose appro- 
priate sensing matrix A in order to achieve good performance 
of (Q]i. A matrix A is said to have the Restricted isometry 
property (RIP) with order k if 



(l-6)\\x\\l<\\Ax\\ 2 2 <(l + 5)\\x\\ 



(2) 



for any vector fc-sparse vectors x. RIP is shown to pro- 
vide stable reconstruction of approximately sparse signals via 
(QJ |5||8|. Moreover, many random matrices satisfy RIP with 
high probability Ifl2l . A matrix A is said to have the Null 
space property of order k (k-NSP) if 



V« e kcrA\{0}, V|T| < k, 



\v T \\l < \\VT" 



NSP is known as a characterization of uniqueness of problem 
(0 when there is no noise J9)- It has also been proven that 
the NSP matrices admit a similar stability result as RIP does 
except that the constants may be larger 0. 

A recent direction of interest in compressed sensing con- 
cerns problems where signals are sparse in an overcomplete 
dictionary D instead of a basis, see 0, HU, 13, ED, ffl. 



This is motivated by the widespread use of overcomplete 
dictionaries in signal processing and data analysis. Many 
signals naturally possess sparse frame coefficients, such as 
images consisted of curves (curvelet frame). In addition, the 
greater flexibility and stability of frames make them preferable 
for practical purposes in order to compensate the imperfectness 
of measurements. In this setting, the signal xq <G C d can be 
represented as xq = Dzq, where zq <E S" and D is a d x n 
matrix with n > d. The columns of D may be thought of 
as an overcomplete frame or dictionary for C d . The linear 
measurements are y = Azo. 

A natual way to recover xq from y is first solving 



are' mm 

zeR n 



ji, subject to y = ADz. 



(3) 



for the sparse coefficients z, then synthesizing it to obtain 
x, i.e., x — Dz. The resulted method is therefore called i 1 - 
synthesis or synthesis based method ifTOl . ifTTI . In the case 
when the measurements are perturbed, we naturally solve the 
following: 

2 = arg min ||^||i, subject to \\y — ADz\\ < e. (4) 

zER n 

The work in IfTTI established conditions on A and D to 
make the compound AD satisfy RIP. However, as pointed 
in 10, iflOl . forcing AD to satisfy RIP or even the weaker 
NSP implies the exact recovery of both zq and xq, which is 
unnecessary if we only care about obtaining a good estimate of 
xq. In particular, they argue that if D is perfectly correlated 
(has two identical columns), then there are infinitely many 
minimizers of (01 that may be assigned to z, but all of them 
lead to the true signal xq. It seems reasonable to expect 
that similar result may hold in the case of highly correlated 
dictionaries, since they are only a small perturbation away 
from the perfectly correlated ones. 

A. Overview and main results 

In this paper, we generalize the ordinary null space property 
to the dictionary case (D-NSP), and prove in Theorem lH.ll that 
this new condition is equivalent to the successful recovery of 
signals in Z?Sfe via I 1 -synthesis. Moreover, a stability result 
is given in Theorem lIII.il To the best of our knowledge, these 
results are the first characterization of compressed sensing with 
dictionaries via I 1 -synthesis approach. 

Section [TV] studies more properties of D-NSP, and shows 
that A has Z3-NSP is equivalent to AD has NSP as long as D 



is full spark (every d columns of D are linearly independent). 
As a consequence, under the full spark assumption, the £ - 
synthesis method cannot accurately recover the signals without 
accurate recoveries of their sparse representations, therefore an 
incoherent dictionary is needed under this circumstance. 

All proofs of the theorems presented can be found in Q, 
while some proofs are provided here. 

II. A SUFFICIENT AND NECESSARY CONDITION FOR 
NOISELESS SPARSE RECOVERY 

In this section, we develop a sufficient and necessary 
condition for the ^-synthesis method (0 to achieve accurate 
reconstruction of sparse signals with noiseless measurements. 
We say the ^-synthesis method (01 is successful in recovering 
Xo when every minimizer z of © satisfies Dz = xq- We show 
that the following property on A is a necessary and sufficient 
condition for successfully recovering all signals in DE S via 
(01, where DY,k — {x : 3 z, such that x = Dz, \\z\\q < k} is 
the set of signals that have fc-sparse representations in D. 

Definition 1 (Null space property of a dictionary D (D-NSP)). 

Fix a dictionary D G C d ' n , a matrix A G (£ m > rf j s sa id to 
satisfy the D-NSP of order k (k-D-NSP) if for any index set 
T with \T\ < k, and any v G D _1 (kcr A\{0}), there exists 
u G kcr D, such that 

\\v T + u\\i < ||t>H|i- (5) 

Theorem II.l. D-NSP is a necessary and sufficient condition 
for the success of ^-synthesis for all signals in the set DEfc. 

Proof: Suppose that ^-synthesis is successful for all the 
signals in DEfc. Take any support T and v G D -1 (kcr A/{0}), 
Let xo = Dvt be the signal that we are trying to recover, 
then by assumption, the minimizer must be vt + u with some 
u G ker D. 

vt — v is also feasible, however, it cannot be a minimizer 
since D(vt — v) ^ Dvt, therefore 

\\v T + u\\i < \\v T - v\\i = ||^||l- 

On the other hand, assuming D-NSP is satisfied, suppose 
by contradiction that Dz ^ Dzq, then v = zq — z G 
D _1 (ker A/{0}), Let T be the support of Zq, therefore there 
exists u G kcr D, \\vr + «||i < ||wt c ||i, i-e. \\z a — z T + u\\ i < 
\\zt4i, so 

lko+«||i < \\z -zt+u\\i + \\z t \\i < Pt-||i + II^t||i = p||i- 

This is a contradiction, since z is the minimizer. □ 
Notice when D is the canonical basis of C d , the Z?-NSP 

is reduced to the normal NSP with the same order. In other 

words, D-NSP is a generalization of NSP for the dictionary 

case. It is, however, a nontrivial generalization. 

The intuition of D-NSP rises from the fact that we are only 

interested in recovering xq instead of the representation zq. 

As long as the minimizer z lies in the affine plane zq + ker D, 

our reconstruction is a success. 



III. D-NSP BASED STABILITY ANALYSIS 

It is known that the NSP is a sufficient and necessary 
condition not only for the sparse and noiseless recovery, but 
also for compressible signals with noisy measurement JT], 
lTT~3l . However, the stability analysis of NSP HI cannot be 
easily generalized to our case because essentially we need the 
function f(v) = (||fr c ||i — K't + «||i)/||Dv||2 to be bounded 
away from zero. In the basis case, we have knowledge of f(v) 
on a compact set, and consequently the extreme value theorem 
can be applied to prove the exisitence of a positive lower 
bound. In our case we do not have a compact set, therefore 
other constructions to overcome this difficulty is necessary. 

Definition 2 (Strong null space property of a dictionary D 
(D-SNSP)). A sensing matrix A is said to have the strong 
null space property with respect to D of order k (k-D-SNSP) 
if for any index set T with \T\ < k, and any v € kcr(AD), 
there exists u € kcr D, such that 

\\vtAW ~ \\ v t + u\\i > c\\Dv\\ 2 (6) 

D-SNSP is a stronger assumption than D-NSP by definition. 
We prove that under this assumption, the i 1 -synthesis recovery 
is stable with respect to perturbations on the measurement 
vector y. 

Theorem III.l. If A is k-D-NSP, then any solution z of 
problem (|4]l satisfies 

\\Dz-x \\ 2 < C^kixo) + C 2 e. 

where ak(xo) denotes the i residue of the best k-term 
approximation to xq, C\, C'2 are constant dependent on n, 
the c in (O, the minimum singular values of A and D, but not 
on Xq. 

Proof: Let xo = Dzo be the true sparse representation. 
Let h = D(z — zq), and we can decompose h as h = Dw + 77 

where Dw G kci'A rj G kcrA^, and H77H2 < — ll^-^lh < 
2e 

— with va being the smallest singular value of A. 

VA 

Let i = D T (DD T )~ 1 r], then 77 = D£, and 

Uh < — Wvh < — e- (7) 

VD V A VD 

Since D{z — Zo) = h = D(w + £), then z — zq = w + ^ + ui 
with some ui G kcrD. 

Let v = w+ux, then we have Dv G ker A and z—zq = v+£. 
So there exists u G kcr D such that © holds, therefore 

\\v + zo,t\U - II ~ u + 2 o,t||i 
>||vt-||i + \\ v t + z 0: t\\i -\\-u t + Zo,t\\i - \\ut-Wi 
>\\v T '\\i - \\v T + u T \\ - \\ut-Wi 

= \\v T c\\i - \\vt + "111 > C II-D U I|2 (8) 
On the other hand, from the fact that I is a minimizer, we get 

|| - U + Z ,t||i + ||z<vH|l >\\-U + Z \\l > \\V + Z + ^j|i 

> \\v + z ||i - HClli > \\v + z , T \\i - ||z ,T=||i - HClli- 



It follows that 

\\v + z ,t\\i - \\-u + zo,t\U < 2||«o,T-|k + ||€||i- ( 9 ) 
Combining ([8]) and (O, we get 

\\Dv\\ 2 < -\\*o,t4i + -M\\i < -\\*o,T4i + —Hh ( 10 ) 
c c c c 

In the end, using dTOb and (0, 

\\h\\ 2 = \\Dv + De|| 2 = \\Dv + f]\\ 2 < \\Dv\\ 2 + \\ V \\ 2 

2 „ \/n., J .„ 1 „ 

< - 20,^ i + — £ 2 + — 2e 

C C 

2„ 2V^ 1 „ 

< - Kt- 1 + — £ + -2e. 

□ 

It is natural to ask how much stronger this new assumption 
is than D-NSP. We address this question partially in the next 
section. 

IV. A FURTHER STUDY OF D-NSP AND ADMISSIBLE 
DICTIONARIES 

This section explores the two assumptions D-NSP and D- 
SNSP further for the purpose of answering the following 
important questions: What kind of dictionaries will allow 
sensing matrices A with few measurements to satisfy D-NSP? 
How to find those sensing matrices given a dictionary? 

We call a d x 77. dictionary D is k-admissible if there exists 
a measurement matrix A £ C m,d with m < d such that A is 
fc-D-NSP. We call D inadmissible if D is not fc-admissible 
for any k > 2. 

The following proposition shows that adding repeated 
columns to the dictionary D will not affect admissibility. This 
is quite intuitive since we do not change the set DEfc during 
this procedure, and we only care about recovering the signal 
xq rather than the representation zq. 

Proposition IV.l. Let D £ C d ' n , and let I be any index set 
I C {l,...,n}. Define D = [D,D/], then for any sensing 
matrix A £ C m '", we have A is D-NSP if and only if A is 
D-NSP. 

Proposition II V. 1 1 states that a perfectly correlated dictionary 
D does not get in the way of the reconstruction of signals. 
It is only natural to ask whether this is still the case for a 
highly coherent dictionary. We answer this question partially 
by showing a class of highly correlated dictionaries is inad- 
missible. Moreover, equivalent conditions of D-NSP is given 
in Section IIV-BI under the assumption that D is full spark. 

A. A Class of inadmissible matrices 

The following theorem constructs a class of inadmissible 
matrices with a dimension 1 kernel. 

Theorem IV.2. Given an orthonormal basis $ = [4>i, ■ 4>d\- 
Let H = span(^j t ) be the union of the hyper- 

I i= {l,...,d}\i 

planes spanned by every combination of d — 1 columns of 



<1>. Then there exists a small constant ro such that for every 
v £ B((f) 1} r )\H, D = [$,v] £ C d ' d+1 is not admissible. 

We need the following lemma for the proof of this Theorem. 

Lemma IV.3. Suppose D is a d x (d + 1) dictionary. If there 
exist Tc{l,...,d+1} with \T\ > 2 such that the normalized 
vector u £ ker D satisfies 

1. ||ut||i > ||«t c ||i> an d 

2. T c C supp(u), 

Then D cannot be \T\-admissible. 

Proof: Assume that D is a dictionary which satisfies the 
assumptions of Lemma IIV.3l and is |T|-admissible at the same 
time. We shall prove this leads to a contradiction. 

For a vector w £ C", we define ||w|| m in = min{|w;j|, i = 
1, ...,n} to be the minimum magnitude in w. Assumption 2 
then implies ||u|| m in > 0. Suppose A is D-NSP and fix a 
V £ D- 1 (ker(A)\{0}). We define a = 2||«b|| 00 /||«|| niJn . 
Now that vq + au,—Vo + au £ D _1 (ker(A)\{0}), we can 
use the definition of D-NSP to derive: there exist c\ , c 2 £ C 
such that 

|| wr + otu T - c\u\\i < \\vtc + mtT<=||i, 

and 

|| — vt + ctu T — c 2 u\\i < || - vt° + auT" ||i, 
Add up the two equations, we get 

\\vt + glut — ciw||i + || — vt + omt — c 2 u\\i 
< \\vtc + au T c ||i + || - v T c + aux'Wi 
= 2a||ur»||i. (ID 

The equality in (fTTT i follows from our definition on a. On the 
other hand, 

\\v T + au T - c\u\\i + || - vt + au T - c 2 u\\i 
= \\v T + (a - ci)u T ||i + |ci|||«t"||i 
+ II - v t + (a - c 2 )u T \\i + |c2|||ur<=||i 
> |2o-ci-cs|||ur||i + (|ci| + |02|)||ur.||i (12) 
(fTTT i and (fT2l together imply 

\2a -a- C3|||«t||i + (|ci| + |cs|)||ure|| x < 2o||«r«||i, 
which can be simplified to 

\\ut\\i < \\ut'\\i, 
This is a contradiction to assumption 1 of Lemma IIV.3I □ 

Proof of Theorem \IV.2\ Notice ker(D) is one dimensional 
and set its basis to be u = (a T ,— 1). Pick an index set T 
with |T| > 2 such that {l,n + 1} £ T. First if v g H, then 
(v,<pi) 7^ for i = l,...,d . This means that all coordinates 
of u are nonzero. Second, we can pick ro small enough such 
that whenever v £ B(<f>\,r), we have ||ut||i > ll u T c ||i- 

Therefore picking v £ B{4>\ , tq)\H fulfills the two assump- 
tions of Lemma IIV.3I hence completes the proof. □ 



Proposition IV.4. If D = [B,v] where B is a full rank d x 
(n— 1) matrix and v = Ba with \\a\\\ < 1, then A has D-NSP 
implies that A has B-NSP with the same order k. 

With this proposition, we can add more columns to the inad- 
missible dictionaries constructed in lIV.2l to obtain inadmissible 
dictionaries with arbitrary dimension. 

B. Relation between D-NSP and NSP 

It is obvious that AD satisfies NSP implies A satisfies 
D-NSP, which explains why imposing RIP or incoherence 
conditions on AD could be too strong and unnecessary. To 
explore how much room there is between these two conditions 
can possibly answer the question whether we can allow 
highly coherent dictionaries or not, since AD being NSP will 
inevitably leads to the incoherence of D. Surprisingly enough, 
we show that whenever D is full spark, these two conditions 
are equivalent. 

A dictionary is full spark means every d columns of this 
matrix are linearly independent. 

Theorem IV.5. The following conditions are equivalent under 
the assumption that D is full spark, 

. A is k-D-NSP; 
. AD is k-NSP; 
. A is k-D-SNSP; 

• For any v £ ker AD, there exists a u such that 
\\v T + u\\i < !|t"H|i- 

Remark IV. 1. Theorem 5.3 implies that for a given full spark 
dictionary D and a given sensing matrix A, if A satisfies D- 
NSP, then all signals x will be recovered by synthesizing the 
already correctly recovered representations z. If A does not 
satisfy D-NSP, although certain signals cannot be recovered 
accurately, there might be signals that are recovered from a 
"wrong" representation. 

In the beginning of Section [Till we mentioned the difficulty 
of proving stability result for D-NSP is due to the non- 
compactness of the set D^ 1 (kcr A\{0}). However, in the full 
spark case, Theorem II V. 5 1 guarantees that we can extend this 
set to its closure, and then the result of Theorem Mill will 
trivially hold under the necessary assumption A being D-NSP. 

We comment that full spark is not a strong assumption on 
matrices. In fact, full spark matrices is dense in the space 
of matrices Q, and a large class of full spark Harmonic 
frames is also constructed in Q. This means that for "most" 
dictionaries, we need to study the composite AD, and AD 
being NSP is the equivalent condition for successful recovery 
of xq £ DYjk- Hence for "most" dictionaries, D is not allowed 
to be very coherent, which is somewhat not expected. 

Remark IV.2. Given an admissible dictionary D that is 
perfectly correlated, we can always find a full spark and highly 
coherent dictionary D that is arbitrarily close to D, therefore 
we cannot find a sensing matrix A such that AD' satisfies k- 
NSP for any k > 2. By Theorem \IV.5\ D' is inadmissible, 



indicating that a small perturbation on the dictionary cannot 
preserve the admissibility. 
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